Multiple Linear Regression¶
- In the previous chapter we learned about predicting the output (Y) for single input (X).
- In this chapter, we are going to learn about predicting the output (Y) for multiple inputs ($X_n$)
Example : Predicting the price of house (Y) on the basis of multiple input : $X_1$ Nummber of bedrooms , $X_2$ age of property , $X_3$ size of property .
Comparision between Single and multiple Linear Regression¶
In 1 input problem $m$ and $c$ are evaluated to represent a line in the 2D space.
In 2 input problem $m_1$ ,$m_2$ and $c$ are evaluated to represent a 2D plane in the 3D space.
For 3 or more input features.¶
-For 3 input problem we need to evaluate ($m_1$ ,$m_2$ ,$m_3$ and $c$ )to represent 3D hyper- plane in a 4dimensional space .
-Similary for more than 3 input features.
-Although it is not visualizable we can evaluate best fit $m_1$ , $m_2$ , $m_3 $ ..... $m_n$ according to the number of input features $X_n$
The process of evalulating best fit $m_1$ , $m_2$ , $m_3 $ ..... $m_n$ and $c$ is same as that of Simple Linear Regression.
$i.e$ Minimizing the Sum of Squared Error.
For one variable $m_1$
$Step 1$ : Evaluation of partial derivative of SSE with respect to $m_1$
Partial Derivative of SSE with respect to $m_1$ is given by :
$P$= $\frac{\partial }{\partial m_1 }SSE$
$P$= $\frac{\partial }{\partial m_1 }\sum(y-(m_1 *x_1+ m_2*x_2+ ...... m_n*x_n +c))^2$
( Here Summation deontes this opeartion to be performed for each data point and all outputs are added )
$P$= $\sum\frac{\partial }{\partial m_1}(y-(m_1 *x_1+ m_2*x_2+ ...... m_n*x_n +c))^2 $
$P$= $ \sum2(y-(m_1 *x_1+ m_2*x_2+ ...... m_n*x_n +c) )(-x_1) $
$Step 2$: Make small step
$new$ $m_1$ = $m_1$ + $k$* $P$
where k is constant called step size. As the name suggests it determine the size of each step .
$Step 3$: Step 1 and 2 are performed for each variables $m_1$ , $m_2$ , $m_3 $ ..... $m_n$ and $c$ .
$Step 4$: Step 1 ,2,3 repetedly until $P$ attends value almost near to zero for each variables $m_1$ , $m_2$ , $m_3 $ ..... $m_n$ and $c$ .